Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POC: Dynamically load ignition and afterburn from rootfs #1834

Draft
wants to merge 1 commit into
base: testing-devel
Choose a base branch
from

Conversation

cgwalters
Copy link
Member

@cgwalters cgwalters commented Jul 8, 2022

This is a proof-of-concept of the idea in
coreos/fedora-coreos-tracker#1247 (comment)

The role of the initramfs originally was just to mount the root filesystem.
Us running ignition from the initramfs makes sense, but it doesn't mean
the ignition binary has to physically live in the initramfs.

In the end state our initramfs for example doesn't need to physically
contain NetworkManager for example either. Or for that matter, kernel network drivers.

It just has to have enough code to mount the root filesystem, and
neither ignition nor afterburn are needed for that.

This clearly adds some nontrivial logic to our already nontrivial
initramfs. But, it does shave 9M (after compression) from each copy of the initramfs,
so in the likely case of having (transiently) 3 different versions,
we will save 27MB in /boot, which is a good amount.

This is a proof-of-concept of the idea in
coreos/fedora-coreos-tracker#1247 (comment)

The role of the initramfs originally was just to mount the root filesystem.
Us running ignition from the initramfs makes sense, but it doesn't mean
the ignition binary has to physically live in the initramfs.

In the end state our initramfs for example doesn't need to physically
contain NetworkManager for example either. Or for that matter, kernel network drivers.

It just has to have enough code to mount the root filesystem, and
neither ignition nor afterburn are needed for that.

This clearly adds some nontrivial logic to our already nontrivial
initramfs.  But, it does shave 9M from each copy of the initramfs,
so in the likely case of having (transiently) 3 different versions,
we will save 27MB in /boot, which is a good amount.
install_ignition_unit ignition-ostree-firstboot-populate-initramfs.service diskful
install_unit ignition-ostree-subsequent-populate-initramfs.service initrd
# Keep this in sync with ignition-ostree-transposefs.sh
rm -v "${initdir}"/usr/bin/{ignition,afterburn}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this hacky bit relies on the fact that we depend on the afterburn and dracut modules, which will have installed the binaries first, then we remove them.

@cgwalters cgwalters marked this pull request as draft July 8, 2022 21:29
@cgwalters
Copy link
Member Author

We could shrink this to only special casing ignition, which would reduce some complexity in dealing with systemd unit dependencies (afterburn-in-initramfs is special in that it runs both before ignition and also after the real root switch).

Also, anything of this form will need special build logic, probably in coreos-assembler to regenerate a "full" initramfs for PXE cases.

@cgwalters
Copy link
Member Author

cgwalters commented Jul 8, 2022

Hmm. Or maybe what would actually be a lot simpler is to build (using dracut) two initramfs images (which we need anyways for PXE), one that e.g. omits ignition and afterburn and NetworkManager etc., and the second which includes them. Our build process could basically diff them, and auto-generate code to copy all the files from the second into the first after mounting the rootfs.

@cgwalters
Copy link
Member Author

cgwalters commented Jul 8, 2022

and auto-generate code to copy all the files from the second into the first after mounting the rootfs.

Though, the ugly part about that is we'd omit the systemd units...so we'd have to do something ugly like a systemctl daemon-reload in the middle of boot. Probably best to only omit ELF binaries which are the big things.

OK so something like:

dracut -f initramfs-full.img
dracut --omit "ignition networkmanager afterburn ..." initramfs-diskful.img
(mkdir full && cd full && lsinitrd --unpack initramfs-diskful.img)
(mkdir diskful && cd diskful && lsinitrd --unpack initramfs-diskful.img)
(cd full && find -type f) | while read file; do 
  if test '!' -f diskful/$file && file ${file} | grep -q ': ELF'; then
    rm diskful/$file
    echo $file >> diskful/etc/removed-manifest.txt 
  fi
done
dracut --initramfs-from-fs diskful initramfs-diskful.img  # regenerate it

Then we have code already in the initramfs which parses /etc/removed-manifest.txt or so and loads from the real root.

@cgwalters
Copy link
Member Author

bgilbert had a notable point in the original issue that relates to this:

We can't further improve this by kicking out NetworkManager and network drivers, since we need those for Tang-bound disk encryption.

Ah, right. That does limit us here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant